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Abstract 


This randomized, double-blind experimental study 
examined the impact of home use of ABCmouse.com 
Early Learning Academy, a comprehensive supplemental 
early learning resource, on literacy and math skills. 
The 13-week study included a diverse sample of 281 
kindergarten-aged children in Southern California. Half 
the children were randomly assigned to use ABCmouse 
for an average of 45 minutes per week at home (Treatment 
group), while the other half did not use ABCmouse (Control 
group). Every four to five weeks, participants completed 
onsite standardized tests that are widely used in schools 
(DIBELS and CBM). While all outcome measures showed 
significant improvements from Time 1 to Time 4, children 
who used ABCmouse showed 25% higher growth on 
a composite of key early literacy skills than children in 
the Control group (p = .01, Cohen’s d effect size = .30). 
The gains from using ABCmouse were even greater 
for children who scored below the sample median at 
pre-test: 33% higher in literacy skills (o < .05, d = .38) 
and 48% higher in math skills (o = .01, d = .49) compared 
to similarly scoring peers in the Control group. These 


findings are especially notable because the intervention 
occurred at home rather than in a classroom setting and 
was of a relatively short duration (13 weeks). 


Participants 


In October 2014, 371 Southern California families 
with children in kindergarten were recruited through 
advertisements in local parenting media and email 
outreach. All families were screened for study eligibility 
requirements: high-speed Internet access on either a 
computer or tablet at home, a kindergarten-aged child 
with some comfort using computers or tablets, and no prior 
ABCmouse exposure at home or school. The sample (46% 
boys, 54% girls) was ethnically diverse (approximately 
15% African American, 15% Asian American, 23% Latino/ 
Hispanic, 48% White). Approximately two-thirds of 
participants (66%) had at least one college-educated 
parent, and about half (48%) had a household income over 
$100,000 per year.' Participants’ assessment scores at 
the beginning of the study were not significantly different 
from the norms for those assessments established with a 
large and nationally representative sample of students.” 


Independent education and program evaluation researcher. Ph.D., Educational Psychology, UCLA; B.A., Psychology, U.C. Berkeley. This research was funded by Age 


of Learning, Inc. The author was responsible for and had full control over study design, research findings, and this report. 


Participating families’ education and income levels were somewhat higher than those of the overall U.S. population, see National Center for Children in Poverty 


(2015; available: http://www.nccp.org/publications/pub_1097.html) and U.S. Census Bureau (2014; available: https://www.census.gov/content/dam/Census/library/ 
publications/2015/demo/p60-252.pdf), but similar to the education and income levels of U.S. families with high-speed Internet access, see Pew Research Center 
(2015, 2013; available: http://www.pewresearch.org/fact-tank/2015/04/20/the-numbers-behind-the-broadband-homework-gap/ and http://www.pewinternet. 


org/2013/08/26/home-broadband-2013/). 


2 DIBELS Next Recommended Benchmark Goals (2012). University of Oregon Center on Teaching and Learning. Available: https://dibels.uoregon.edu/docs/ 
DIBELSNextRecommendedBenchmarkGoals.pdf; Saven, J. L., Tindal, G., Irvin, P. S., Farley, D., & Alonzo, J. (2014). EasyCBM Norms 2014 Edition. University of Oregon 
Behavioral Research and Teaching. Available: https://dibels.uoregon.edu/docs/techreports/Math_TechRpti409_easyCBM_Norms.pdf. 
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The final sample included 281 children (128 in the 
Treatment group and 153 in the Control group); attrition 
was primarily due to failure to meet the 45-minute per 


Design & Procedure 


Before the first assessment, children were randomly 
assigned to either the Treatment or Control group. Trained 
educators who were blind to condition administered 
the DIBELS (Dynamic Indicator of Basic Early Literacy 
Skills) Next Assessment? and the CBM (Curriculum 
Based Measurement Solutions) Lite Numbers and 
Operations? individually to each child at four time points 
between October 2014 and January 2015 at a market 
research facility. 


In order to maintain fidelity of the instruments, children 
were given a different version of each DIBELS and CBM 
Benchmark and Progress Monitoring assessment at each 


week ABCmouse usage requirement in the Treatment 
group, use of ABCmouse in the Control group, and 
voluntary withdrawal from the study.? 


time point. At the conclusion of each assessment, all 
families received a gift card. 


Table 1 shows the skills assessed. Because of differences 
in scale of the literacy variables, an average of the 
standardized scores (z scores) of LNF, FSF, PSF, and NWL- 
CLS were used as the DIBELS Literacy Composite. To 
calculate the z scores, each literacy score was centered 
at the mean of the sample at Time 1 to preserve changes 
in scores over time, enabling direct comparison of the 
Treatment and Control groups at each time point and on 
their learning gains over time.° 


Table 1. Skills Assessed by the DIBELS and CBM Benchmark and Progress Monitoring Assessments 


Assessment 


After the Time 1 assessment, Treatment group families 
received a 20-minute training session with instructions 
for their child to use ABCmouse for 45 minutes per week 


w 





Skills Assessed 


during the 13-week study. Although all children in the final 
sample used ABCmouse for an average of at least 45 
minutes per week, no child met the usage requirement 


Attrition occurred for the following reasons: Treatment group—insufficient ABCmouse usage (46 children), voluntary withdrawal from study (8), refusal to take 


assessment (2); Control group—ABCmouse usage during study (19), voluntary withdrawal from study (12), refusal to take assessment (3). Importantly, the Treatment 
and Control groups in the final sample were baseline equivalent, with similar demographics and no significant differences in literacy or math scores at the 


beginning of the study (Time 1). 


4 Good, R. H., & Kaminski, R. A. (Eds.) (2007). Dynamic Indicators of Basic Early Literacy Skills (6th ed.). Eugene, OR: Institute for the Development of Educational 


Achievement. Available: http://dibels.uoregon.edu/. 


5 Hosp, Michelle K. (2007). The ABCs of CBM: A practical guide to curriculum-based measurement. New York: Guilford Press. 


€ Achild’s Literacy Composite (z score) at Time 1 shows their relative place within the sample. A child at the mean of the full sample at Time 1 would have a score 
of O and a child below the mean would have a negative score. Literacy Composite scores at other time points represent changes in standard deviation units from 
the mean at Time 1. A child with a score of 0.5 at Time 4 performed 0.5 standard deviations higher at Time 4 than the mean of the full sample at the beginning of 


the study. 
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every week of the study. Treatment group children in more than 75 minutes per week. Control group families 
the final sample used ABCmouse for an average of 82 completed a weekly survey documenting their child’s 
minutes per week, with over three-quarters averaging online activities. 


more than 60 minutes per week and half averaging 


Results 
Finding 1 


Children who used ABCmouse achieved significantly greater gains on key early literacy skills than children who did 
not use ABCmouse. 


While all outcome measures showed statistically significant improvement over the 13-week study period (Time 1 to Time 
4), children who used ABCmouse showed 25% higher growth on the DIBELS Literacy Composite than children in the 
Control group (d = .30; Figure 1). Intent-to-treat analyses including the Treatment group children who did not meet the 
required ABCmouse usage threshold confirmed this result (p < .05, d = .29). 


Literacy Gains 
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Figure 1. Growth in DIBELS Literacy Composite from Time 1 to Time 4. On gain score, t(277) = 2.56, p = .01, d = .30. An analysis of covariance 
(ANCOVA) on the gain score with Time 1 as a covariate also confirmed this main effect of condition, F(1,276) = 4.78, p < .05. 
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Finding 2 


ABCmouse particularly helped lower-performing children improve their literacy and math skills. 


The ABCmouse Treatment group children who scored below the sample median at Time 1 achieved 33% higher gains 
in literacy (d = .38; Figure 2) and 48% higher gains in math (d = .49; Figure 3) by Time 4 than their peers in the Control 


group who also scored below the median at Time 1. 
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Literacy Gains for Lower-Performing Children 
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Figure 2. Time 1 and Time 4 scores on the DIBELS Literacy Composite for children who scored below the median at Time 1 (N = 139). On gain 


scores, t(137) = 2.21, p < .05, d= .38. 


Math Gains for Lower-Performing Children 
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Figure 3. Time 1 and Time 4 scores on the CBM Numbers and Operations for children who scored below the median at Time 1 (not z scores, 


N = 103). On gain scores, t(101) = 2.50, p = .01, d = .49. 


Conclusion 


This randomized controlled trial demonstrates that the 
use of ABCmouse at home can significantly accelerate 
the academic growth of kindergarten-aged children. With 
a 13-week intervention, the Treatment group achieved 
25% higher growth on a composite of important early 
literacy skills than the Control group. Notably, the literacy 


and math gains were even greater for Treatment group 
children who underperformed at pre-test (33% and 48% 
higher, respectively). Using ABCmouse at home can help 
these struggling students improve, providing parents a 
way to supplement other remedial classroom work. 
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